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Abstract 

While  the  RGB ,  HSV  and  YUV  (YCbCr)  are 
standard  models  used  in  various  colour  imaging 
applications,  not  all  of  their  information  are  necessary 
to  classify  skin  colour.  This  paper  presents  a  novel  skin 
colour  model,  RGB-H-CbCr  for  the  detection  of  human 
faces.  Skin  regions  are  extracted  using  a  set  of 
hounding  rules  based  on  the  skin  colour  distribution 
obtained  from  a  training  set.  The  segmented  face 
regions  are  further  classified  using  a  parallel 
combination  of  simple  morphological  operations. 
Experimental  results  on  a  large  photo  data  set  have 
demonstrated  that  the  proposed  model  is  able  to 
achieve  good  detection  success  rates  for  near-frontal 
faces  of  varying  orientations,  skin  colour  and 
background  environment.  The  results  are  also 
comparable  to  that  of  the  AdaBoost  face  classifier. 

Keywords:  Face  detection,  skin  colour,  colour  models, 
skin  classification,  skin  modeling. 

1.  Introduction 

Detection  of  the  human  face  is  an  essential  step  in 
many  computer  vision  and  biometric  applications  such 
as  automatic  face  recognition,  video  surveillance, 
human  computer  interaction  (HCI)  and  large-scale  face 
image  retrieval  systems.  The  first  step  in  any  of  these 
face  processing  systems  is  the  detection  of  the  presence 
and  subsequently  the  position  of  human  faces  in  an 
image  or  video. 

The  main  challenge  in  face  detection  is  to  cope  with 
a  wide  variety  of  variations  in  the  human  face  such  as 
face  pose  and  scale,  face  orientation,  facial  expression, 
ethnicity  and  skin  colour.  External  factors  such  as 
occlusion,  complex  backgrounds,  inconsistent 
illumination  conditions  and  quality  of  the  image  may 
also  contribute  significantly  to  the  overall  problem. 

Throughout  the  last  decade,  there  has  been  much 
development  in  face  detection  research,  particularly  in 


the  abundance  of  methods  and  approaches.  Recent 
surveys  [1],  [2]  have  comprehensively  reviewed 

various  face  detection  methods  available  in  the 
literature. 

Face  detection  in  colour  images  has  also  gained 
much  attention  in  recent  years.  Colour  is  known  to  be  a 
useful  cue  to  extract  skin  regions,  and  it  is  only 
available  in  colour  images.  This  allows  easy  face 
localisation  of  potential  facial  regions  without  any 
consideration  of  its  texture  and  geometrical  properties. 

Most  techniques  up  to  date  are  pixel-based  skin 
detection  methods  [3],  which  classifies  each  pixel  as 
skin  or  “non-skin”  individually,  and  independently 
from  its  neighbours.  Early  methods  use  various 
statistical  colour  models  such  as  a  single  Gaussian 
model  [4],  Gaussian  mixture  density  model  [5],  and 
histogram-based  model  [6] . 

Some  colour  spaces  have  their  luminance 
component  separated  from  the  chromatic  component, 
and  they  are  known  to  possess  higher  discriminality 
between  skin  pixels  and  non-skin  pixels  over  various 
illumination  conditions.  Skin  colour  models  that 
operate  only  on  chrominance  subspaces  such  as  the  Cb- 
Cr  [7],  [8],  [9]  and  H-S  [10]  have  been  found  to  be 
effective  in  characterising  various  human  skin  colours. 

Skin  classification  can  be  accomplished  by 
explicitly  modelling  the  skin  distribution  on  certain 
colour  spaces  using  parametric  decision  rules.  Peer  et 
al.  [11]  constructed  a  set  of  rules  to  describe  skin 
cluster  in  RGB  space  while  Garcia  and  Tziritas  [12] 
used  a  set  of  bounding  rules  to  classify  skin  regions  on 
both  YCbCr  and  HSV  spaces. 

In  this  paper,  we  present  a  novel  skin  colour  model 
RGB-H-CbCr  for  human  face  detection.  This  model 
utilises  the  additional  hue  and  chrominance  information 
of  the  image  on  top  of  standard  RGB  properties  to 
improve  the  discriminality  between  skin  pixels  and 
non-skin  pixels.  In  our  approach,  skin  regions  are 
classified  using  the  RGB  boundary  rules  introduced  by 
Peer  et  al.  [11]  and  also  additional  new  rules  for  the  H 
and  CbCr  subspaces.  These  rules  are  constructed  based 
on  the  skin  colour  distribution  obtained  from  the 


training  images.  The  classification  of  the  extracted 
regions  is  further  refined  using  a  parallel  combination 
of  morphological  operations.  The  rest  of  the  paper  is 
organised  as  follows:  Section  2  briefly  describes  the 
various  steps  of  our  face  detection  system.  The 
construction  of  the  RGB-H-CbCr  skin  colour  model  is 
described  in  Section  3.  Section  4  presents  the  use  of 
morphological  operations  in  our  algorithm. 
Experimental  results  and  discussions  are  provided  in 
Section  5.  Finally,  Section  6  concludes  the  paper. 

2.  System  Overview 


Training  Images  Input  Image 


Detected  Faces 


Figure  1.  System  overview  of  face  detection 
system 

Fig.  1  shows  the  system  overview  of  the  proposed 
face  detection  system,  which  consists  of  a  training  stage 
and  detection  stage.  In  our  colour-based  approach  to 
face  detection,  prior  formulation  of  the  proposed  RGB- 
H-CbCr  skin  model  is  done  using  a  set  of  skin-cropped 
training  images.  Three  commonly  known  colour  spaces 
-  RGB,  HSV  and  YCbCr  are  used  to  construct  the 
proposed  hybrid  model.  Bounding  planes  or  rules  for 
each  skin  colour  subspace  are  constructed  from  their 
respective  skin  colour  distributions. 

In  the  first  step  of  the  detection  stage,  these 
bounding  rules  are  used  to  segment  the  skin  regions  of 
input  test  images.  After  that,  a  combination  of 
morphological  operations  are  applied  to  the  extracted 
skin  regions  to  eliminate  possible  non-face  skin 
regions.  Finally,  the  last  step  labels  all  the  face  regions 
in  the  image  and  returns  them  as  detected  faces.  In  our 
system,  there  is  no  preprocessing  step  as  it  is  intended 
that  the  input  images  are  thoroughly  tested  under 
different  image  conditions  such  as  illumination 


variation  and  quality  of  image.  Since  we  emphasise  on 
the  use  of  a  novel  skin  colour  model  in  this  work,  our 
system  is  restricted  to  colour  images  only. 

3.  RGB-H-CbCr  Model 

3.1.  Preparation  of  Training  Images 

In  order  to  build  the  skin  colour  model,  a  set  of 
training  images  were  used  to  analyse  the  properties  and 
distribution  of  skin  colour  in  various  colour  subspaces. 

The  training  image  set  is  composed  of  140  skin 
colour  patches  of  ten  colour  images  obtained  from  the 
Internet,  covering  a  wide  range  of  variations  (different 
ethnicity  and  skin  colour).  These  images  contained  skin 
colour  regions  that  were  either  exposed  to  normal 
uniform  illumination,  daylight  illumination  (outdoors) 
or  flashlight  illumination  (under  dark  conditions).  The 
skin  regions  in  all  ten  training  images  were  manually 
cropped  for  the  purpose  of  examining  their  colour 
distribution.  Fig.  2  shows  two  samples  of  the  skin- 
cropped  training  images. 


Figure  2.  Skin-cropped  training  images 
3.2.  Skin  Colour  Subspace  Analysis 

The  prepared  skin  colour  samples  were  analysed  in 
the  RGB,  HSV  and  YCbCr  spaces,  As  opposed  to  3-D 
space  cluster  approximation  used  by  Garcia  and 
Tziritas  [12],  we  intend  to  examine  2-D  colour 
subspaces  in  each  of  the  mentioned  colour  models,  i.e. 
H-S,  S-V,  H-V  and  so  forth,  to  model  the  skin  clusters 
more  compactly  and  accurately. 

In  RGB  space,  the  skin  colour  region  is  not  well- 
distinguished  in  all  3  channels.  A  simple  observation  of 
its  histogram  will  show  that  it  is  uniformly  spread 
across  a  large  spectrum  of  values. 

In  HSV  space,  the  H  (Hue)  channel  shows 
significant  discrimination  of  skin  colour  regions,  as 
observed  from  the  H-V  and  H-S  plots  in  Fig.  3  where 
both  plots  exhibited  very  similar  distribution  of  pixels. 
In  the  hue  channel  shown  in  Fig.  4,  most  of  the  skin 
colour  samples  are  concentrated  around  values  between 
0  and  0.1  and  between  0.9  and  1.0  (in  a  normalized 
scale  of  0  to  1). 
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HSV  Skin  color  distribution  H  vs  V 


HSV  Skin  color  distribution  H  vs  S 


Figure  3.  H-V  and  H 


-S  subspace  plots 


Figure  4.  Distribution  of  the  H  (Hue)  channel 


Figure  5.  Distribution  of  Y,  Cb  and  Cr 

Some  studies  have  indicated  that  pixels  belonging  to 
skin  regions  possess  similar  chrominance  (Cb  and  Cr) 
values.  These  values  have  also  been  shown  to  provide 
good  coverage  of  all  human  races  [8].  Our  analysis  of 
the  YCbCr  space  using  our  training  set  further 
substantiates  these  earlier  claims,  and  have  shown  that 
the  Cb-Cr  subspace  offers  the  best  discrimination 
between  skin  and  non-skin  regions.  Fig.  5  shows  the 
compact  distribution  of  the  chrominance  values  (Cb 
and  Cr)  in  comparison  with  the  luminance  value  (Y). 

It  is  also  observed  that  the  varying  intensity  values 
of  Y  (Luminance)  does  not  alter  the  skin  colour 
distribution  in  the  Cb-Cr  subspace.  The  luminance 
property  merely  characterises  the  brightness  of  a 
particular  chrominance  value. 

3.3.  Skin  Colour  Bounding  Rules 

From  the  skin  colour  subspace  analysis,  a  set  of 
bounding  rules  is  derived  from  all  three  colour  spaces, 
RGB,  YCbCr  and  HSV,  based  on  our  training 


observations.  All  rules  are  derived  for  intensity  values 
between  0  and  255. 

In  RGB  space,  we  use  the  skin  colour  rules 
introduced  by  Peer  et  al.  [11].  The  skin  colour  at 
uniform  daylight  illumination  rule  is  defined  as 

(R  >  95)  AND  (G  >  40 )  AND  (B  >  20)  AND 

(max{R,  G,  B}  -  min{R,  G,  B}  >  15)  AND 

(\R  -G\>  15)  AND  (R  >  G)  AND  (R  >  B)  (1) 

while  the  skin  colour  under  flashlight  or  daylight  lateral 
illumination  rule  is  given  by 

(R  >  220)  AND  (G  >  210)  AND  (B  >  170)  AND 
(\R  -  G\  <  15)  AND  (R  >  B)  AND  (G  >  B)  (2) 

To  consider  both  conditions  when  needed,  we  used 
a  logical  OR  to  combine  both  rule  (1)  and  rule  (2).  The 
RGB  bounding  rule  is  denoted  as  Rule  A. 

Rule  A:  Equation(l)  u  Equation(2)  (3) 

Based  on  the  observation  that  the  Cb-Cr  subspace  is 
a  strong  discriminant  of  skin  colour,  we  formulated  5 
bounding  planes  from  its  2-D  subspace  distribution,  as 
shown  in  Fig.  6.  The  five  bounding  rules  that  enclosed 
the  Cb-Cr  skin  colour  region  are  formulated  as  below: 


Cr<  1.5862xCb +  20  (3) 

Cr  >  0.3448  x  Cb  +  76.2069  (4) 

Cr  >  -4.5652  x  Cb  +  234.5652  (5) 

Cr  <  -1.15  xCb  +  301.75  (6) 

Cr  <  -2.2857  x  Cb  +  432.85  (7) 


Rules  (3)  to  (7)  are  combined  using  a  logical  AND 
to  obtain  the  CbCr  bounding  rule,  denoted  as  Rule  B. 
Rule  B:  Equation(3)  n  Equation(4)  n  Equation(5)  n 
Equation(6)  n  Equation(7)  (8) 

In  the  HSV  space,  the  hue  values  exhibit  the  most 
noticeable  separation  between  skin  and  non-skin 

regions.  We  estimated  two  cutoff  levels  as  our  H 

subspace  skin  boundaries, 

H  <  25  (9) 

H  >  230  (10) 

where  both  rules  are  combined  by  a  logical  OR  to 
obtain  the  H  bounding  rule,  denoted  as  Rule  C. 

Rule  C:  Equation(9)  u  Equation(lO)  (11) 

Thereafter,  each  pixel  that  fulfills  Rule  A,  Rule  B 
and  Rule  C  is  classified  as  a  skin  colour  pixel, 

Rule  A  n  Rule  B  n  Rule  C 


(12) 


Skin  color  distribution  Cb  vs  Cr 


Figure  6.  Bounding  planes  for  Cb-Cr  subspace 


Figure  7.  Skin  segmentation  using  the  RGB-H 
CbCr  model 


3.4.  Skin  Colour  Segmentation 

The  proposed  novel  combination  of  all  3  bounding 
rules  from  the  RGB,  H  and  CbCr  subspaces  (Equation 
12)  is  named  the  “RGB-H-CbCr”  skin  colour  model. 
Although  skin  colour  segmentation  is  normally 
considered  to  be  a  low-level  or  “first-hand”  cue 
extraction,  it  is  crucial  that  the  skin  regions  are 
segmented  precisely  and  accurately.  Our  segmentation 
technique,  which  uses  all  3  colour  spaces  was  designed 
to  boost  the  face  detection  accuracy,  as  will  be 
discussed  in  the  experimental  results.  Fig.  7  shows  the 
skin  segmentation  result  of  two  sample  test  images 
using  the  RGB-H-CbCr  model. 

The  resulting  segmented  skin  colour  regions  have 
three  common  issues: 

a)  Regions  are  fragmented  and  often  contain  holes 
and  gaps. 

b)  Occluded  faces  or  multiple  faces  of  close 
proximity  may  result  in  erroneous  labeling  (e.g.  a 
group  of  faces  segmented  as  one). 

c)  Extracted  skin  colour  regions  may  not  necessarily 
be  face  regions.  There  are  possibilities  that  certain  skin 
regions  may  belong  to  exposed  limbs  (arms  and  legs) 
and  also  foreground  and  background  objects  that  have 
a  high  degree  of  similarity  to  skin  colour  (also  known 
as  false  alarms). 


4.  Morphological  Operations 

The  next  step  of  the  face  detection  system  involves 
the  use  of  morphological  operations  to  refine  the  skin 
regions  extracted  from  the  segmentation  step. 

Firstly,  fragmented  sub-regions  can  be  easily 
grouped  together  by  applying  simple  dilation  on  the 
large  regions.  Hole  and  gaps  within  each  region  can 
also  be  closed  by  a  flood  fill  operation. 

The  problem  of  occlusion  often  occurs  in  the 
detection  of  faces  in  large  groups  of  people.  Even  faces 
of  close  proximity  may  result  in  the  detection  of  one 
single  region  due  to  the  nature  of  pixel-based  methods. 
Hence,  we  used  a  morphological  opening  to  “open  up” 
or  pull  apart  narrow,  connected  regions. 

Additional  measures  are  also  introduced  to 
determine  the  likelihood  of  a  skin  region  being  a  face 
region.  Two  region  properties  -  box  ratio  and 
eccentricity  are  used  to  examine  and  classify  the  shape 
of  each  skin  region. 

The  box  ratio  property  is  simply  defined  as  the 
width  to  height  ratio  of  the  region  bounding  box.  By 
trial  and  error,  the  good  range  of  values  lie  between  1.0 
and  0.4.  Ratio  values  above  1.0  would  not  suggest  a 
face  since  human  faces  are  oriented  vertically  with  a 
longer  height  than  width.  Meanwhile,  ratio  values 
below  0.4  are  found  to  misclassify  arms,  legs  or  other 
elongated  objects  as  faces. 

The  eccentricity  property  measures  the  ratio  of  the 
minor  axis  to  major  axis  of  a  bounding  ellipse. 
Eccentricity  values  of  between  0.3  and  0.9  are 
estimated  to  be  of  good  range  for  classifying  face 
regions.  Though  this  property  works  in  a  similar  way  as 
box  ratio,  it  is  more  sensitive  to  the  region  shape  and  is 
able  to  consider  various  face  rotations  and  poses. 

Both  the  box  ratio  and  eccentricity  properties  can  be 
applied  to  the  extracted  skin  regions  either  sequentially 
or  parallelly,  following  a  dilation,  opening  or  flood  fill. 

5.  Experimental  Results  and  Discussion 

The  proposed  RGB-H-CbCr  skin  colour  model  for 
skin  region  segmentation  was  evaluated  on  a  face 
detection  system  using  a  test  data  set  of  100  images, 
containing  a  total  of  600  unique  faces.  In  order  to  build 
this  test  data  set,  the  images  were  randomly  selected 
from  the  Internet1,  each  comprising  of  two  or  more 
near-frontal  faces  and  of  a  large  variety  of  descent 
(Asians,  Caucasians,  Middle-Eastern,  Hispanic  and 
African).  The  test  images  also  consist  of  various  indoor 

lA\\  images  taken  from  the  Internet  were  used  solely  for  this  research 
only  and  there  were  no  attempts  to  re-use,  edit,  manipulate  or 
circulate  these  images  for  any  other  interests. 


and  outdoor  scenes  and  of  different  lighting  conditions 
-  daylight,  fluorescent  light,  flash  light  (from  cameras) 
or  a  combination  of  them.  The  size  of  each  image 
ranged  from  500x350  to  1000x600,  regardless  of  face 
size.  The  face  detection  system  was  implemented  using 
MATLAB  on  a  2.4MHz  Pentium  IV  machine  running 
on  256  MB  RAM. 

To  evaluate  our  experiments,  we  defined  two 
performance  metrics  to  gauge  the  success  of  our 
schemes.  False  Detection  Rate  (FDR)  is  defined  as  the 
number  of  false  detections  over  the  number  of 
detections. 


FDR  = 


false  detections 
number  of  detections 


x  100% 


Detection  Success  Rate  (DSR)  is  defined  as  the  number 
of  correctly  detected  faces  over  the  actual  number  of 
faces  in  the  image. 


DSR  =  ‘ 


correctly  detected  faces 
number  of  faces 


x  100% 


where  the  number  of  correctly  detected  faces  is 
equivalent  to  the  number  of  faces  minus  the  number  of 
false  dismissals. 

Table  1  shows  the  experimental  results  of  the  face 
detection  system  using  various  combination  of 
morphological  operations  (as  described  in  Section  4). 
The  face  detection  system  achieved  a  good  detection 
rate  of  90.83%  using  a  parallel  combination  of 
opening,  box  ratio  and  eccentricity  operators.  Other 
combinations  resulted  in  a  poorer  DSR  or  higher  FDR. 

The  proposed  scheme  was  also  compared  with  the 
well-known  AdaBoost  face  detector/classifier  by  Viola 
and  Jones  [13],  and  results  showed  that  the  proposed 
scheme  (with  the  right  configuration  of  morphological 
operators)  is  able  to  reach  comparable  standards  to  that 
achieve  by  the  AdaBoost  algorithm  (90.17%)  on  the 
similar  data  set. 

To  evaluate  the  effectiveness  of  the  RGB-H-CbCr 
skin  colour  model  (Table  2),  the  face  detection  system 
was  tested  with  various  combination  of  colour  models, 
each  represented  by  its  own  set  of  bounding  rules.  The 
combination  of  ah  3  subspaces  resulted  in  the  best  DSR 
and  lowest  FDR  values.  Fig.  8  presents  some  sample 
results  of  the  proposed  face  detection  system. 

As  shown  in  the  experimental  results,  the  proposed 
method  sometimes  failed  to  detect  a  face  correctly,  as 
seen  from  the  high  FDR  of  28.29%  (Table  1).  This 
could  be  attributed  to  the  usage  of  morphological 
operators.  Though  these  operators  are  used  parallelly  to 
improve  the  likelihood  of  detecting  faces,  it  may 
sometimes  cause  “over-detection”  of  faces.  Fig.  8(b) 
and  Fig.  8(c)  show  examples  of  false  detections  and 
false  dismissals  encountered  in  our  experiments. 


Table  1.  Face  detection  experimental  results 


Method 

FDR 

(%) 

DSR 

(%) 

Opening+BoxRatio+Eccentricity 

(parallel) 

28.29 

90.83 

AdaBoost  [13] 

18.65 

90.17 

Opening+B  oxRatio 

20.76 

87.17 

Opening+Eccentricity 

23.73 

85.17 

Dilation+Opening 

42.29 

83.00 

Opening+B  oxRatio+Eccentricity 
(sequential) 

13.36 

82.17 

Table  2.  Results  using  various  combination  of 
colour  model  bounding  rules _ _ 


Method 

FDR  (%) 

DSR  (%) 

RGB  only  [11] 

43.05 

69.00 

RGB+CbCr 

36.14 

77.17 

RGB+H 

33.82 

83.50 

RGB+H+CbCr 

28.29 

90.83 

The  RGB-H-CbCr  skin  color  model  is  able  to  deal 
with  various  brightness  and  illumination  conditions,  but 
it  remains  susceptible  to  detection  of  non-skin  objects 
that  possess  similar  chrominance  levels  as  skin  colour. 

Occlusion  remains  a  difficult  problem  to  tackle, 
especially  when  colour  is  used  as  the  cue  for 
segmentation.  Occluded  faces  and  faces  that  are  closely 
located  are  often  merged  together  by  a  minute  portion 
of  a  connected  skin  part.  Fig.  8(d)  shows  an  example  of 
misclassification  of  multiple  persons  due  to  occlusion. 

6.  Conclusion 

In  this  paper,  we  have  presented  a  novel  skin  colour 
model,  RGB-H-CbCr  to  detect  human  faces.  Skin 
region  segmentation  was  performed  using  a 
combination  of  RGB,  H  and  CbCr  subspaces,  which 
demonstrated  evident  discrimination  between  skin  and 
non-skin  regions.  The  experimental  results  showed  that 
our  new  approach  in  modelling  skin  colour  was  able  to 
achieve  a  good  detection  success  rate.  On  a  similar  test 
data  set,  the  performance  of  our  approach  was 
comparable  to  that  of  the  AdaBoost  face  classifier. 

In  future  work,  we  intend  to  refine  the  use  of 
morphological  operations  in  the  post-processing  of  the 
extracted  skin  regions.  An  adaptive  training 
(incremental  learning)  of  the  skin  colour  model  can  be 
used  to  improve  the  overall  classification  of  skin 
regions.  Primarily,  the  elimination  of  false  detections 
and  false  dismissals  is  crucial  to  the  success  of  a  robust 
face  detector. 


(d) 


Figure  8.  Sample  face  detection  results  using 
the  test  data  set 
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